Pursue Robust Indefinite Scalability

For research insights and development potential, we should explore computer architectures designed to scale indefinitely. Given physical limits, we argue an indefinitely scalable computer should or must (1) reveal to programmers its component spatial relationships, (2) forego unique addresses, and (3) operate asynchronously. Further, such a machine and its programming must be inherently robust against local failures and outages, and be operable during its own construction. We propose the indefinitely scalable Movable Feast Machine, which defers many architectural decisions to an execution model that associates processing, memory, and communications functions with movable bit patterns rather than fixed locations. We illustrate basic and novel computational elements such as self-healing wire, simple cell membranes for modularity, and robust stochastic sorting by movable self-replicating programs. 1 Indefinite scalability and robustness Although computation has scaled largely by increasing data widths and switching speeds, progress has stalled at present; multicores are currently ascendant, but cache coherence scales poorly. Achieving indefinite scalability—i.e., supporting open-ended computational growth without substantial re-engineering—will involve new distributed system architectures, and impact hardware, OS design, programming, and end-user value propositions. So although strategies such as multicore still have room to run, we view indefinite scalability as a useful beacon now, casting light into some still largely wild computational spaces beyond the serial machine. A design for an indefinitely scalable machine amounts to a spatial tiling of hardware elements, with additional requirements as needed to preserve open-ended physical realizability. Three particularly consequential ones are: 1. Pledge allegiance to the light cone: Because hardware occupies space and light speed is finite, no fixed-cost global communications or arbitrary long range links are possible, ruling out hierarchical and other log time interconnects, at least above some granularity—and also implying asynchronous or dynamically-synchronizing operation. 2. with computational relativity: There must be only one or a few basic hardware functional types, interchangeable by kind, and without presumption of any globally unique identifying attributes1. Therefore, inter-element addressing depends, at least initially, upon relative spatial position. 3. and robustness for all: The system should provide non-stop operation, tolerating or repairing (or even exploiting) errors and disruptions, such as transients and misconfigurations, hot-swapping and new construction, etc. This applies beyond hardware to the entire computational stack—consistency and correctness cannot simply be assumed, lest a local error, however unlikely, derail the entire system. Existing tiled hardware such as [9, 13] generally emphasizes finite (e.g., intra-chip) scalability, but may be useful components for indefinitely scalable designs; in any case software looms as a major challenge. Indeed, to imagine programming without perfect reliability and synchronization may seem simply crippling, but we have found it can also feel liberating and full of natural potential— more like how things actually get done in the world, even though stuff breaks, or people deliver late, or never, or deliver white and call it wheat. Society often uses systems that tolerate many such failures and perform useful work nonetheless, preferring minor inefficiencies to major catastrophes. From the vantage point of striving to add value despite the inevitable errors, it is traditional computing’s demand for absolute perfection in input and processing—or else “It’s not my fault!”—that seems petty and bureaucratic, like that bad apple with whom nobody wants to work. To keep on growing, from bottom to top, what computation needs is team spirit. 1A potentially contentious issue is whether each hardware element can be expected to offer an independent (pseudo) random stream.